Rater Training to Support High-Stakes Simulation-Based Assessments

On Rater Agreement and Rater Training

This paper first analyzed two studies on rater factors and rating criteria to raise the problem of rater agreement. After that the author reveals the causes of discrepencies in rating administration by discussing rater variability and rater bias. The author argues that rater bias can not be eliminated completely, we can only reduce the error to a cetain degree by training raters. The study on r...

Simulation-based training improves physicians' performance in patient care in high-stakes clinical setting of cardiac surgery.

BACKGROUND Simulation-based training is useful in improving physicians' skills. However, no randomized controlled trials have been able to demonstrate the effects of simulation teaching in real-life patient care. This study aimed to determine whether simulation-based training or an interactive seminar resulted in better patient care during weaning from cardiopulmonary bypass (CPB)-a high stakes...

Enriched rater training using Internet based technologies: a comparison to traditional rater training in a multi-site depression trial.

OBJECTIVE The evaluation and training of raters who conduct efficacy evaluations in clinical trials is an important methodological variable that is often overlooked. Few rater training programs focus on teaching and assessing applied clinical skills, and even fewer have been empirically examined for efficacy. The goal of this study was to develop a comprehensive, standardized, interactive rater...

Simulation as a high stakes assessment tool in emergency medicine

The Australasian College for Emergency Medicine (ACEM) will introduce high stakes simulation-based summative assessment in the form of Objective Structured Clinical Examinations (OSCEs) into the Fellowship Examination from 2015. Miller's model emphasises that, no matter how realistic the simulation, it is still a simulation and examinees do not necessarily behave as in real life. OSCEs are suit...

High-Stakes Interpretation